Assessing family medicine trainees – what can we learn from the European neighbours?

Background: Although demands on family physicians (FP) are to a large extent similar in the European Union, uniform assessment standards for family medicine (FM) specialty training and assessment do not exist. Aim of this pilot study was to elicit and compare the different modalities and assessment methods of FM specialty training in five European countries. Methods: A semi structured survey was undertaken based on a convenient sample in five European countries (Denmark, Germany, Poland, the Netherlands and the United Kingdom). The respondents were asked to respond to ten items about aspects of FM specialty training and assessment methods in their respective countries. If available, this data was completed with information from official websites of the countries involved. Results: FM specialty training is performed heterogeneously in the surveyed countries. Training time periods range from three to five years, in some countries requiring a foundation program of up to two years. Most countries perform longitudinal assessment during FM specialty training using a combination of competence-based approach with additional formative and summative assessment. There is some evidence on the assessments methods used, however the assessment method used and costs of assessment differs remarkably between the participating countries. Conclusions: Longitudinal and competence-based assessment is the presently preferred approach for FM specialty training. Countries which use less multifaceted methods for assessment could learn from best practice. Potential changes have significant cost implications.


Background
The family physician (FP) is the first medical contact for prevention, acute or chronic diseases and provides longitudinal continuity of comprehensive care in almost all European health care systems [1]. In a changing European Union (EU) with its growing population mobility and doctor migration, continuous health care adaptions are inevitable [2], [3]. Facing these social and economic changes, the European countries have the opportunity to learn from each other [4]. The European Academy of Teachers in General Practice and Family Medicine de-veloped the European Educational Agenda for family medicine (FM) in 2005, defining core characteristics of the discipline and essential competences [5]. This agenda was intended to be incorporated into the national curricula for FM specialty training according to the needs of the particular health care system. Unfortunately, this was not achieved homogenously throughout Europe as several countries are using a different framework model for competency based training [6]. Within the last decades competence-based training has been regarded as an important development within medical education. Compared to traditional education programs, competencebased training aims to be outcome-orientated, analyzing functional occupational roles and assessment of the trainees' progress. In principle, the competence-based approach could be more flexible, transparent and standardized [7], [8]. The competence-based approach in medical training has already been implemented in European countries such as the Netherlands (NL) and the United Kingdom (UK). Other countries such as Germany (DE) are currently introducing a competency approach for future FM specialty training [6]. So far, there are no studies which examine and compare international ways of qualification and assessment for family medicine. The aim of this study was to carry out a pilot study aiming to raise different modalities and assessment methods of FM specialty training in five European countries.

Methods Questionnaire
A study group in Heidelberg consisting of three FPs and one FP trainee developed a 10 item semi structured questionnaire addressing aspects of postgraduate training and qualification for FPs. Within these 10 items, prerequisites for admission of examinees to assessment, qualifications of the examiners, evidence for assessment methods and costs of these methods were investigated. The questions were all open-ended and in English. Table 1 shows the questionnaire used. Following the professional code of conduct of State Medical Chamber of Baden-Württemberg, for this survey no approval by an institutional research review board was needed [http:// www.aerztekammer-bw.de/10aerzte/40merkblaetter/ 20recht/05kammerrecht/bo.pdf cited 2014 August 12]. Recruitment: The pilot survey was conducted in five European countries based on a convenient sample linked to postgraduate training programs in Denmark (DK), Germany (DE), Poland (PL), the NL and the UK. Target participants for this study were identified due to relationships established via an European implementation research network named "Tailored implementation for chronic diseases" (TICD) [http://www.ticd.umed.lodz.pl/ index.php/homepage cited 2014 August 12]. Four researcher of the TICD project were asked to suggest a person in their country who is a well-established experts in the field of FM specialist training. To also include the representation of the European Academy of Teachers in General Practice / Family Medicine (EURACT) a council member of EURACT was additionally asked to participated in the study. Five persons participate in the pilot study, all were FPs and actively involved in FM training and assessment in their respective country. All correspondence was done by email.

Data analysis
Data from the questionnaire was analyzed by two researchers from the study group in Heidelberg independently from one another. This extracted data was collated, categorized and then summarized and discussed resulting in a consensus version of results. This version was again double checked by the surveyed participants and the study group. If applicable, information like costs of assessment was expanded from official websites of the participant countries to ensure that the information given was generalizable  August 12]. Special emphasis was set on assessment methods and organization of FM specialty training. The enhanced version of results was then revised by each member of the study group in Heidelberg to ensure correct reproduction of data, resulting in a final version, which provided a descriptive overview of FM specialty training and assessment methods in the different countries. All indicated amounts of money were converted into US dollars (US$) according to currency exchange rates on 30.01.2014.

Results
All five experts in the field of FM specialist training agreed to participate. The duration of FM specialty training in the participating countries ranges from three to five years. All FM specialty trainings include hospital and practice training in approved training posts. In DK and the UK, a foundation program -a compulsory medical training part in between graduation and start of FM specialty training -is mandatory. As the FM training is highly favored in the UK and in the NL, a national selection process before training was implemented in the UK and soon will be in the NL. The contents of FM specialty training are defined by FP specialist representatives at national level in all surveyed counties but DE, where all medical specialties not only FPs determine the details. In all participant countries, the assessment is carried out and results are accepted nationwide. All countries had a , which is continuously carried out through specialty training. These three parts combine multiple assessment methods as computer based MCQ exams and the use of electronic devices in testing, as well as assessment of competencies and procedural skills in different settings as live observation and video consultation with "real" or simulated patients. Special emphasize is set on feedback from colleagues and the patients' perspective. Trainees have to keep a detailed e-portfolio, which includes a learning log, a personal development plan, regular self-ratings and educator reviews. In the NL and the UK there is robust evidence for the reliability and validity of the different assessment methods, whereas there is less evidence for the assessment methods used in DE, DK and PL [9], [10], [11], [12], [13], [14], [15]

Discussion
This pilot study gives an overview of specialty training and different qualification assessment methods for FM in five European countries. Assessment methods differ remarkably in all surveyed countries. All countries but DE perform longitudinal assessment during FM specialty training using a combination of a competence-based approach with additional formative (formal and informal assessment methods during the whole period of training learning, "assessment for learning") and summative assessment (formal assessment of outcome, summarization learning progress at a particular time, "assessment of learning"). DK and the NL assess all required competencies longitudinally in an ongoing process during the whole training, including in the NL communication skills. The process of qualification for FM in the UK is highly structured. In contrast, DE uses no longitudinal assessment and only one final theoretical oral assessment at the end of specialty training. Evidence published to date validates the knowledge test, the communication skills test and the competency assessment list, "Compass" in the NL [9], [10], [11], [12], [13] and assessment methods in the UK [14], [15], [[http:// www.rcgp.org.uk cited 2014 August 12]. Otherwise, many FM assessment methods have been reported in scattered surveys, although final conclusions or recommendations are yet to emerge [16]. As outlined previously, population mobility and physician migration in the EU presents new challenges for health care. For quality and patient safety reasons, competence profiles and future assessment methods in the EU might preferably be standardized [2], [17]. If this were to be the case, competencies requirements could still differ  between countries and individuals, but an exchange of physicians and trainees between countries would be more effectively facilitated. In particular DE, performing the least multifaceted assessment of all surveyed countries, could follow the different European examples and implement a longitudinal assessment of competences and skills in FM specialty training. In DE there is evidence, that nearly every fourth of the FM-trainers do give informal feedback less than once per month to their trainees [18]. It should be taken into consideration that critics claim that the competence-based approach neglects expertise and raises negative pressure on examiners and examinees [16]. So far, little evidence exists that performance in the different assessment methods (MCQ, video or live observation, OSCE, etc.) correlates with clinical skills in real working conditions [19], [20], [21]. Therefore, countries with rather elaborate specialty training may continuously reflect on their methods of assessment. There is evidence that assessment influences learning [22], [23], therefore assessment may be an instrument for the regulation of content of FM specialty training and should match population health needs as well as learning objectives and curriculum. To assess the individual level of professional competence, a multifaceted and complex assessment has to be performed [24]. Preferably, the assessment should be an integrated part of the curriculum using a blend of methods [25]. As shown in the results, structured and standardized assessment methods are considerably more costly than less multifaceted approaches, differing up to more than US$ 3.000 per examinee. Therefore, in each European country, available personal and financial resources have to be considered, too. It remains a subject for discussion how much FM specialty training may cost and to what extend examinees may be charged. However, FM trainers should receive professional recognition including adequate reimbursement. The strength of the study is that it is a first overview of different methods of specialty training and qualification assessment for FM in Europe. The pilot survey provides baseline data on an area that is likely to become more topical in the context of EU policy. A limitation of the study is the convenience sampling. However, it was in line with the aim of the study to gather and compare baseline information on the variability and diversity of FM specialty training and assessment methods within the participating countries. The questionnaire contained furthermore some closed-ended questions, which may have led to limited answers. In addition, the data collected in the sample was heterogeneous. This was somewhat challenging for summarizing in the results section. Moreover, through the "coding" process of summarizing answers to the openended questionnaire, some subjective bias may occur. To minimize this effect, all authors thoroughly reviewed the transcript manuscript. Finally, evidence-based information is not available for all questions in the questionnaire, which may lead to expert-based conclusions being drawn.
Next work package will be to build a questionnaire with the results of this pilot-study to raise quantitative data about FM specialty assessment throughout Europe.

Conclusion
This pilot study provides a first overview of different methods of FM specialty training and assessment methods in Europe. Longitudinal and competence-based approaches are presently favored in developed European countries. Countries as DE, which perform less multifaceted assessments can benefit from best practice in neighboring countries. Of critical importance to future FM specialty training programs is the financial aspect: assessing competencies is more costly than assessing knowledge. More research is needed, to establish whether performance in assessment correlates with clinical skills in real working conditions and the new challenges occurring in primary health care service delivery.